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Abstract: The problem of minimizing mean response time of generic jobs 
submitted to a heterogenous distributed computer systems is considered in 
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The article is closely related to a previous article on the same topic. The 
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provides a new formulation, and discusses the impact of new findings, based 
on the improved formulation, on the results of the previous article. 
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1 Introduction 



Recent years have been witness to an increasing use of distributed computing 
system. This may be attributed to two main factors: growth of the Internet, 
and low cost solution of end-user computing devices. Many business processes 
such as supply chain management are distributed due to the inherent nature 
of tasks involved with them. Besides, scale of economy is often possible due 
to the use of clusters of less powerful computers instead of a central computer 
of significantly high power. However, a distributed solution can yield the true 
advantage only if it is possible to distribute works evenly among the processors 
(terms processor, host and computer are used inter-changeably in this article). 
In other words, when load on the computers in a distributed environment 
has significant variance of workloads, high performance can be achieved by 
redistributing loads. The task of redistributing the loads on the computers is 
called load balancing. 

Load can be characterized as jobs or tasks. A job indicates a complete and 
independent entity of work to be completed. However, a job may consist of 
number of sub-units called tasks, and for the job to be completed the compris- 
ing tasks need to often communicate among themselves. For simplification, in 
this article, we consider mutually independent tasks units as jobs. Jobs can 
be classified into two broad categories - dedicated and generic. Dedicated jobs 
can be processed only on specified hosts, while generic jobs can be processed 
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on any host in the system. Once a generic job is submitted to the system, it 
can be processed at the place of its origin or can be transferred it to another 
processor. Irrespective of the processor chosen, once the job is started at a 
processor, it remains there till completion. Some of the most commonly used 
performance indicators of a load balancing algorithm are mean response time, 
throughput, variance of response time. In this article, we confine our analysis 
based on mean response time only. 

Algorithms or heuristics for load balancing fall into two categories: static and 
dynamic. Static load balancing techniques do not depend on the state (charac- 
terized by workload etc.) of the processors. Simple static load balancing tech- 
niques such as join-the-shortest queue (SQ), minimum expected delay (MED) 
have been in use for over a decade. The SQ policy allocates an arriving task 
to the processor having the minimum load and, the MED policy allocates an 
arriving task to the processor having the minimum expected value of waiting 
time for currently scheduled tasks. Note that though these two policies may 
work similarly for homogenous processors, their behaviors differ for hetero- 
geneous processors. Tantawi and Towsley have proposed an optimization 
model and load balancing algorithms to determine static optimal allocation 
of loads among the hosts connected in a broadcast network. Ross et al. Q 
considered a more general problem consisting of dedicated and generic jobs, 
and also dealt with scheduling decision at each host. The authors have noted 
that the problem is separable over local scheduling decisions, and suggested a 
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solution procedure based on this finding. 

Dynamic load balancing techniques, on the other hand, rely on present and 
past states of the processors. In dynamic load balancing, though an initial 
assignment of work to the processors is done through static load balancing, 
subsequent adjustment of work depends on the workload profile of the proces- 
sors. Dynamic load balancing is expected to have higher overhead, but better 
performance than static load balancing. Most of the dynamic load balancing 
algorithms have two important components: Information exchange through 
which processors exchange their load information, and subsequent load ex- 
change to redistribute loads. Depending on the characteristics of these com- 
ponents, there can be different types of dynamic load balancing ( [[jj 0, ||, 
i). 

However, in this article we concentrate on static load balancing. We express 
some of the points of our disagreement with || and provide alternatives. The 
rest of the paper is organized as follows. Section 2 suggests an improved prob- 
lem formulation. The solution procedure is illustrated in Section 3. Concluding 
remarks are provided in Section 4. Points of disagreement are marked in italics. 

2 Improved Problem Formulation 

We adopt the model of distributed computing as proposed in . In this model 
host computers are treated as node. Each node i G {1, 2, • • -n} is capable of 
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executing any incoming job k. Arrival rate of job at node i is (pi, and total 
arrival rate of jobs on the system is $ = Y%=i <t>i- When a job k arrive at a 
node i, it may be executed at node i itself or transferred to another node j, 
depending on the distribution of loads at that time. If job is transferred to 
node j, it is executed at node j without being transferred to any other host, and 
results of execution is transferred back to node i so that the user submitting 
the job gets the illusion of the job being executed at the host i. Processing 
rate of job at node i is $ and transfer rate of jobs from node i to node j is Xij. 
Therefore, A = £™ =1 £?=i the total traffic on the network. 

It is also assumed that hosts may be heterogeneous (different CPU speed, 
memory and swap availability etc.) and speed of execution depends on the 
instantaneous load of the host. Therefore the instantaneous marginal delay 
due to local processing of a job k at host i depends on host i as well as load 
on the host i, and is denoted by Fi(/3i). So the mean response time of all the 
jobs on the network due to local processing is given by £™ =1 v « f „ Fi(f3i) = 

In the distributed set up, another major component of mean response time 
is the network delay. The incremental communication delay for transfer of a 
packet can be modeled as (^(x), where x is the vector of load transfer. Note 
that we are assuming that the network delay due to sending the response 
back is negligible. In case it is not so, the argument of G should be suitably 
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replaced. Q So the contribution of network delay to mean response time is 
given by £?=i £?=i ^^ G„(x). 

[It is to be noted that in as well as in j^tf, denominator used was $ which 
is not equal to J27=i 2j=i x ij ■ Hence, their formulations did not use average 
communication delay, contrary to their claims. Also the claim in ||/ that x in 
G(x) can include response packets is wrong as in that case total flow no longer 
remains A.] 

Hence the problem of load balancing can be formulated as: 
Minimize 

no n n 

E ^P-ffm) + E E V n fin Ggfr) 

subject to 

n n 

i=l i=l 
n n 

A + E x ii = fa + E x i*> ^ = 1, 2, • ■ ■ , n 

n n 

x = EE^-, 

i=l j=l 

A > 0, z = 1,2, ••■,n, 
* If we assume that is the amount of flow of results from node i to 

node j, then a more accurate model would have network delay given by: 

Ya=i £?=i — t — \Gij(x + y), where y is the vector of additional traffic 

Li=iLj=i( 3;i i" , "W') 

due to response packets. 
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Xij > 0, i,j = 1,2, ■ ■ -,71, 
•^ii 0; i 1) 2, • • • , n, 



(1) 

Note that first two constraints ensure that all incoming load is processed, 
and flow balance holds true. As in ||, we assume that F»(/3i) are increasing 
convex functions and Gy(x) is a non- decreasing convex function. Ethernet 
and satellite are quite common network infrastructure. Due to the type of 
data transmission means on these, the network delay for transfer of a packet 
from node i to node j depends on the total network load. Henceforth, we also 
assume that GV,(x) depends on the total traffic A on the network but not 
on any component of it. So the second term in the objective function can be 
written as £?=i £™ =1 ^Gy(A). 

3 Solution of the Problem 

Solution to the above problem can be expressed in terms of disjoint set of 
nodes. We classify the nodes as source (idle or active), neutral and sinks as 
in 0]. A node i is said to be source if Xjt = Vj and 3j x^- > 0. A source 
i is idle if /3j = 0, else it is active. A node i is said to be sink if = Wj 
and 3j > 0. For a neutral node i, x^j = Vj and Xji = Vj. We denote 
the set of sinks, idle sources, active sources and neutrals by S, Rd, R a and 
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respectively. 



Tantawi and Towsley |J claimed that triangular inequality of network delays 
(i.e., Gij(X) < Gik(X) + Gkj(X)) was sufficient condition for solution of the 
problem as formulated by them to yield only source, sink and neutral nodes. 
We illustrate in the following theorem that the claim is wrong. 

Theorem 1 The triangular inequality of network delay and non-decreasing 
property of are sufficient conditions for Problem to yield nodes which 
are either sources, sinks or neutrals. 

Proof: For a given incoming load (3 is constant, and hence we just need to 
show that the stated conditions are sufficient for a flow rate assignment x with 
nodes either sinks, sources or neutral nodes to minimize the communication 
term C = J27=i Ej=i Let us suppose that i(x) denotes the number of 

nodes that are neither sinks, sources nor neutrals for the flow rate assignment 
x. Therefore we need to show that ?(x) = 0. Of all the flow rate assignment 
minimizing communication delay, choose the flow rate assignment x which has 
maximum z(x) and it is strictly positive. [In Proof of Theorem 1 in ^j, x was 
chosen to have minimum i(x) from candidate solutions minimizing communi- 
cation delay. And it was proved that z(x) = 0. But this does not rule out the 
possibility that there can be other optimal solutions with z(x) > 0. So it only 
proves that there is at least one flow rate assignment minimizing the objective 
function for which nodes are either sources, sinks or neutrals. ] Since i(x) > 0, 
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there is at least one node k and a pair / and m such that x\ k > and x km > 0. 
Consider a new assignment x' which differs from x only in the following. 

x ik = x ik ~ min{xi k , x km }, 

X km = X km ^-^{Xlki x km\i 

x lm = xi m + min{x ifc , X km }. 
Let the new Communication delay be C . So, 

n n rpl 

i=ii=i A 

/ Glk(\ ) , G km (X') , Gl m (X') /V^ Gij[X' 



X lk \i ' X km \i ' X lm \ , ■ / , / , './ x 

AAA i=lj=1 A 



F + (££ X ij w I ij"> 3 

) & (l,k), (k,m), (l,m)}). 



Since A' < A and is non-decreasing, 

r» {„> Gik(X') Gik{X) x G km (X') G km (X) ^ 

° — ° ^ \ X lk v Xlk ^ J rtm -y x fem ^ ; 

j / Gi m (X') Gi m (X) 1 , , . Gi k (X) 

+ \ X lm ^ J — l X iA; — X lk) T 

/ , _ n Gkm{X) , , _ s Glm(X) 

'\ X km x km) » + \ X lm x lm) > 

= ^[min{:rz fc ,£ fcm }(G7 fc (A) + G km (X) - G im (A))] < 0. 



Therefore i(x') < i(x), a contradiction. 

[in Proof of Theorem 1 in with new x' fotaZ network load no longer remains 
X. Also it is possible to have z(x') = z(x) — 2, and no£ always z(x') = z(x) — 
1).] □ 
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As for broadcast network, delay can be assumed to be independent of source- 
destination pair, Gij(X) is independent of i and j. Moreover, since Yh=i fli = 
Y^=i 4>i an d total incoming load is constant, the objective function in Prob- 
lem [TJ is equivalent to 

i=i i=i 

For simplifying the solution procedure, let Ui = J2]=iXji,Vi = Y?j=i%ij- So 
Problem |l| can be written as: 

Minimize 

n n n 

i=l i=l i=l 

subject to 

Pi + Vi = 4>i + Ui, % = 1, 2, • • • , n, 

n n 

J^Vi -J^iii = 0, i = 1, 2, • • • , n 

i=l i=l 

A > 0, i = 1,2, ■ • - ,n, 
Ui > 0, i — 1, 2, • • • , n, 
> 0, z = 1, 2, • • • , n. 

(2) 

Note that Y%=i A = J27=i 0i = $ is constant. After eliminating the first 
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constraint, problem 0has the following form. 



Minimize 



E(«i ~ v i + <t>i)Fi{ui -Vi + (pi) + ®G(J2 V i) 

j=l i=l 



subject to 



n n 



E Vi - E Ui = 0, i = 1, 2, • • • , n 

i=l i=l 

Ui- + > 0, i = 1,2,- • - ,n, 
> 0, i = 1, 2, • • • , n, 
t?j > 0, z = 1, 2, • • • , n. 

(3) 

Theorem 2 The optimal solution to the above problem can be expressed as: 

fi((3i)>a + /3G\X), f3i = 0(ie R d ), 
fi(Pi) = a + PG'(X), < A < fa (ie R a ), 
a < f0i) <a + 0G'(X), A = 0, (ie N), 
fi{/3i) = a, Pi> 4 (ieS), 

subject to total flow constraint, 



E /rV + ®g>(\)) + e ^ + E /r x («) = 
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where 



A = E(/r 1 («)-0,)- 



Proof: The Lagrangean function for this problem is 

n n 

L(u, v, a, 7, if;, rj) = J2( u i ~ v i + <fii) F i(ui -Vi + (pi) + ®G(J2 v *) 

n n n n n 

+a(E Vi-J2 u i) + Yl 7»(«i - Vi + 4>i) + ^i U i + E 

1=1 1=1 i=l i=l i=l 

It can be shown easily that objective function and the constraints satisfy 
conditions for applying Karush-Kuhn- Tucker (KKT) conditions. Let fi(Pi) = 
Fi(Pi) + f3i Fi ^ ■ Applying KKT conditions, we get the following set of equa- 
tions. 

SL 

7— = - Vi + 00 - a + 7i + V»i = 0, i = 1, 2, • • • , n, (4) 
dUi 

8L n 

-j- = ~fi(ui -Vi + (pi) + $G"(E v i) + a - li + Vi = °, * = !> 2, • • • , n, (5) 

^ = -E^ + E^ = o, (6) 

i=l i=l 

Ui- Vi + (pi> 0, 7j(ui - + (pi) = 0, 7i < 0, % = 1, 2, • • • , n, (7) 

«i > 0, ipiiii = 0, ^ < 0, z = 1, 2, • • • , n, (8) 

Vi > 0, 77^ = 0, rn < 0, i = 1, 2, • • • , n. (9) 

Let us consider two cases separately, ut — Vi + (pi = and «j — Vi + 0j > 0. 
Case 1: Ui — Vi + (pi = 0, i.e., $ = 0. 
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Again we consider two sub cases: 
1A: fa > 0. 

In this case Vi > 0, and consequently from Equation |9] rji = O.So from Equa- 
tion |5] 

fi(Pi) > a + pG'(\). 

Also for this case, f3{ = 0, fa > 0. It can easily be noticed that these are idle 
source nodes. 

IB: fa = 0. 

It is obvious that in this case Ui = i>j. From Equation [| and Equation [5], 

$G'(A)+ 7i + ^ = 0. 

But by assumption $G"(A) > and from Equation |7| and Equation |9] 7i < 
0, r\i < 0. This implies atleast one of 7 « and r]i is strictly negative, and corre- 
spondingly from Equation ^ or Equation |^ either it, or Vi is zero. Therefore 
Ui = Vi = 0. Hence, from Equation ^| > a. Also /3j = 0, 0j = 0. 

These nodes correspond to neutral nodes without external load. 

Case 2: Ui — Vi + fa > 0. 

From Equation |7|, 7i = 0. Now consider three subcases: 

13 



2A: Vi > 0, ui = 0. 

Since > 0, from Equation |9| rji = 0. Hence from Equation ||, we get 

f l {(3 i ) = a + (3G\\). (10) 

For this case, < Pi < fa. Therefore these nodes correspond to active sources. 
2B: Vi = 0,Ui> 0. 

Since Ui > 0, from Equation ^ ipi = 0. Hence from Equation ^, 

fi{Pi) = a. 

In this case Pi > fa. These are sink nodes. 
2C. Ui = Vi = 0. 

Since ^ < 0, from Equation f| we get fi(Pi) > a. Similarly, since rji < 0, from 
Equation || we get fi(Pi) < a + /3G'(X). Here Pi = fa, and hence nodes are 
neutrals. 

To solve the system completely, we need to find out value of a. From Equa- 
tion |6|, Yn=i Pi — Substituting values of Pi from the solution nodes in the 
expression we get EieR a fi rl ( a + $G'(\)) + Eiejv fa + EieR a fi rl ( a ) = We 
still need value of A to solve the above equation to find out a for the incum- 
bent solution. Fortunately A = J2i=i u i = J2ies u i = J2ies(Pi + v i ~ 4>i) = 
J2i£s(Pi ~ 4>i) = J2ies(fi rl ( a ) ~ 4>i)- Now we are at a position to find out a 
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from the above equation. 

[ There are some mistakes in the proof of corresponding theorem in j^j which 
we presume to be typographical error.] □ 

The result of the above theorem can be intuitively explained as illustrated be- 
low. All the sink nodes have the lowest and equal incremental node delay (a). 
Loads are shared among the sink nodes such that all have equal incremental 
node delay. For all the active source nodes the incremental node delay equals 
sum of incremental node delay of sink nodes and total incremental communi- 
cation delay at the present load {(3). If the incremental node delay at a source 
is greater than this value then it is profitable, from the perspective of the 
objective to reduce mean response time, to transfer all the load of the source 
to the sink nodes, and these nodes correspond to idle source nodes. Note that 
load distribution among the active source nodes are such that all of them have 
equal incremental node delay(a + (3G'(\)), and these nodes process portion of 
their incoming load ((pi) transferring the remaining loads to sinks. However, if 
at a node the incremental node delay is less than the above value, then it is 
not profitable to send any portion of the load to sink node and thereby incur 
communication delay. All the incoming load at such a node is processed by 
itself, and it is a neutral node. The above solution is soothing to practitioners' 
eyes also. The optimal solution suggests a distribution such that nodes with 
lower incremental node delay are utilized more extensively than others. 
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4 Conclusion 



This article points out a number of inconsistencies in ||, and improves on 
them. One of the nice aspects of the results in the present article is that 
the parametric analysis as well as single-point load balancing algorithm as 
suggested in f| can be applied with suitable modification of expressions from 
the solution given in the previous theorem. It is also perceivable that due 
to the simplicity of the expression, the present procedures can not be more 
difficult than those in [?]. 

Fortunately, the present finding does not affect the results of [|J. As mentioned 
before, Ross et al. had considered a general problem consisting of generic as 
well as dedicated jobs, and included the task of scheduling jobs at each host. 
The authors showed that problem was separable over scheduling decisions. 
They also showed that given an allocation of the jobs on the hosts, the task of 
scheduling can be solved as a polymatroid optimization problem. The present 
results in this paper only changes the allocation of the jobs on the hosts and 
does not violate any of the assumptions made in ||. 

As noted in ||, dynamic load balancing can yield better response time. Most 
of the dynamic load balancing tool uses a decision making unit (called load 
balancer) which periodically monitors load on hosts. Also hosts can decide 
to inform of any exigency when load on them exceed a high level threshold 
or goes below a low level threshold (both these thresholds can be dynamic). 
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Load balancer then decides on new load distribution and executes the transfer. 
The present result can be used for dynamic load balancing. The expected 
values of a and a + (3G'{\) can serve as lower and higher thresholds. The load 
balancer can keep track of incremental node delay (a function of allocated 
load on the node. See ]3| for some illustrative measures of load) of hosts 
as well as incremental communication delay. When it receives any receiver- 
initiated request for a load, generated by load on the host going below the 
lower threshold), it can invoke single-point algorithm to reallocate the load. 
Similarly, it can redistribute load when it receives any sender- initiated request, 
generated by load on the host going above the upper threshold. 
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